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ABSTRACT 

An overview of educational evaluation theory in 
general, and its formative phase in specific, is presented. 
Educational evaluation is defined as are its formative and summative 
phases and its classifications of external and internal evaluation. 
Each phase and classification are discussed largely in the context of 
formative evaluation. In addition, various key aspects of formative 
evaluation are discussed: key problems, sources of information, 
activity phases^and types of formative evaluation. (Author/BH) 
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Introduction 

"Data for decision-making" has appropriately been sug- 
gested as the slogan for educational evaluators. Its desig- 
nation is quite obvious. The emphasis has been and continues 
to be that "...every type of data collected will contribute 
to the decision-making process itself." (1) Educational 
evaluation then finds its modus operandi in the milieu of 
data and decision making. Fittingly, Stufflebeam and the 
Phi Delta Kappa National Study Committee on Evaluation defined 
educational evaluation as "the process of delineating, obtaining, 
and providing useful information for judging decision alter- 
natives." (6) Thus, educational evaluation is a very complex 
and comprehensive process of gathering, assimilating, and 
synthesizing data which in turn is used to adjust, discard, 
or otherwise change the application of an on-going educational 
process. 

Three processes, then, emerge. They are the evaluation 
process, the decision making process, and the educational 
process. In the arena of a local educational system in which 
each of these processes is operational and fully functioning, 
theoretically speaking, an equilibrium will exist in^^ich 
each is complementary of and exists solely for the benefit of 
the others. In essence, in order for this system tP so 
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function the evaluation process must provide accurate and 
valid data zh, in turn, is utilized in the decision making 
process, and results in an on-going educational process. 

A process paradigm forms as shown below. The evaluation 
process as symholized in the "box, is the fooal point for this 
paper; the interrelationship of the three processes will "be 
examined as appropriate. Note that only the evaluation and 
decision making processes are connected "by two solid lines. 



Evaluation 
Process 



Decision Making 
Process 




X. Educational 
Process 



Process Paradigm of the Evalttation, Decision 
Making, and Educational Processes, 
Note I The solid lines with their directional 
indicators illustrate direct interaction 
"between direct processes; the dashes with their 
directional indicators illustrate an indirect 
interaction "between processes. 

The educational process, although not at all isolated from 
the other two, has "both solid and dashed lines connecting it 
with the decision making process and also similar lines running 
from the educational to the evaluation process; the dashed 
lines flow in the opposite direction. (This flow indicates 
only an indirect relationship hetween .he evaluation and the 
educational process while there is very much a direct rela- 
tionship hetween the decision making and educational processes. 
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The relationship is in terms of final authority and regulatory- 
power. The evaluation unit rarely has any program regulatory 
authority of a direct nature, "but definitely exerts an indirect 
influence according to the institutionalized chain of command. 

Formative and Summative Evaluation 

Educational evaluation, especially when considered in the 
context of a local school system, has been classified into 
two phases. (5) They are designated formative and summative 
evaluation. Since any process can be divided into phases, the 
developmental phase of a program or curricula of the educational 
process is rightly referred to as formative, and the final or 
completion phase of a program or curricula of the educational 
process is called summative. More specifically, formative 
evaluation refers to the process of judging a dynamic product 
that can be revised in form since it is still undergoing 
development. Conversely, svimmative evaluation is the process 
of judging a completed product which has been readied for the 
consumer (i.e., the students, teachers, and administrators) 
and thus final validation. (1, pp. 390-391) 

According to Lawson (3), the major purpose of formative 
evaluation is to "proffer descriptive and judgmental data 
which enhance rationality in decision making relevant to the 
design of instructional products." The purpose of summative 
evaluation can be worded exactly the same, but with the last 
phase reading "relevant to the validation of the finalized 
design of Instructional products." From these ^purpose 



statements it can be inferred that the overall purpose of 
educational evaluation is to derive reliable data for (re) 
designing and/or validating instructional products or curricula, 
Lawson (3) has also advised that "information presented to 
the learner or the instructor only for the conduct of an on- 
going instruction is not formative." a?hus, so-called evalu- 
ation efforts which only provide curriculum helps or instruc- 
tional suggestions really cannot be classified as data provided 
from a true evaluation endeavor. 

Internal and External Evaluation 

Closely related to the formative and summative phases of 
educational evaluation are two classifications of educational 
evaluation. They are external and internal evaluation. Both 
classifications are based upon the source of the evaluation 
effort. Quite simply, external evaluation refers to formative 
or sxammative evaluation by an evaluation team employed from 
outside the local school system; internal formative or summa- 
tive evaluation refers tp an evaluation team employed as an 
integral unit of the local school administrative staff, (1, 
pp, 402-403) 

0?he trend of the 1970' s appears to be a polarization of 
internal and external evaluation in terms of the two phases 
of educational evaluation. Most formative evaluatipns of 
local ^ucational systems and programs 
the jurisdiction of an instit\iti6nalized evaH^atip 
while most summative evaluations of ^^ocsdl'J^y^t 
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are being intrusted to an external team financed by the local 
school system or by a state education agency, a university, 
or the U. S. Office of Education. 

However, various large municipal school systems, such as 
the Dallas (Texas) Independent School District and the Columbus 
(Ohio) Public Schools, have both formative and summative evalu- 
ation phases established on an internal continuing basis. An 
example of this comprehensive approach to educational evalu- 
ation can be seen by an examination of the goals and areas of 
service delineated by the Evaluation Department of the Columbus 
(Ohio) Public Schools, The stated goals of this evaluation 
department are (^)i 

To contribute to the improvement of programs, 
services, operation, and administration of the 
Columbus (Ohio) Public Schools. 

To contribute to the improvement of communi- 
cation between and among the Board of Education, 
employees of the Board, pupils, parents and 



citizens. 



15: . groups of pupils. 



To help the Board of Education and school admin- 
istration account to pupils, parentis, knd citizens 
for the performance of the entire school and 



1^ . To create, maintain, arid improve oppor^ 

for pupils, parents, and citizens to have a 
broader and more meaningful inyolyeirient in^ 

ii'-"'' -school, affairs.;/;. ;;-.,; 

■ • To reflect opinions and ^t^^^^ 

parents.^ and citizeris?^^^ 
iife ! the Board of 

; ■ • To reinf orcerv 

^#b-A/'"- v.* and vihfoi^atl^^^^ 




utilized by "both federal or state funding agencies as well as 
provide immense service to the local education agency decision 
makers. According to Merriman (^), internal evaluation teams 
have the added advantage of providing for continuity and 
building a cumulative asset in the local school system. 

Aspects of Formative Evaluation 

Since most reviews of literature concerning educational 
evaluation focus largely upon theory, the emphasis will be 
upon practice in terms of the practical, everyday application 
of current evaluation theory. This pragmatic approach is 
appropriate in that the central theme of this study is upon 
formative evaluation efforts as undertaken in the setting of 
the local education agency. This transference of theory into 
practice will be discussed in connection with several impor- 
tant aspects that confront any internal formative evaluation 
team. These aspects arei key problems, sources of informa- 
tion, plus activity phases and types of formative evaluations. 

Kgy Problems and Factors 

Every internal formative evaluation component of a local 
school system is confronted with problems. These problem^ — „_ 



may range from the one extreme in which an evaluation team 
has to deal with a not-atr-all unusual situation in wwh 
small suburban school business manager- is in charge of curri-T^^rMf|f|| 
culum and its selection.. At the other, eartreme;^ an" ©valuation ' r' ^J^l 
team in' a • large munic ipal school system mi^t. have the prpbleii^y^ 
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of developing and implementing a new social studies program 
for a recently desegregated student body that is being 
bused amid strong parental protest. Most internal formative 



|- evaluation problems stem from innovations, implementations, 

and impedimenta. In other words, innovations must be con- 
ceived for new or developing programs, these in turn must be 
implemented, and any impedimenta that results from the inno- 
vations and their implementations must be overcome. Five key 
problem areas can be identified. They are the problem areas 
I ' of institutionalization, of decision making, of role deline- 

It ation, of the time-frame, and of evaluation models^pjid methodo- 

logies. (7i2) Since each of these key problem areas, if 
examined thoroughly, would necessitate lengthy discussions, 
certain pertinent factors related to these areas can be iso- 
lated. The^e factors are examined in the context of the 
initial institutionalization of a formative evaluation unit 
in a local school system, and are based on a paper by Walker. ^ C^^^ 
The installer of the formative evaluation system should ■ 
be sure that the prevailing definition of the system is con- 
i - si stent with his expectations for that system, his ability 

p| ■ and competence to install such a system,- and his philosophy. 

as to what-ends- should be served by an eyalAjatl^ 

The evaluator must know who the decision makers are . 
The most common and most hazardous way of ■/identi;^in^■^them. 5^;.^.^-|;;./|^ 
to''- ic)bk ■■■at 'orgaoiizatiSKSiT^s^^ 

■■■■■■•..■■■■.■■■^■■■■:. ^:#t1K;af||p|tfc 
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decision makers are compounded since the decision making 
structure is jTluid and the structure always operates on "both 
the formal and the informal levels. In addition there are 
different types of power (action and veto, direct and indirect). 

In designing an evaluation system, the evaluator*m\;^st 
delineate information sources, methods of retrieval, and 
techniques for providing the information. An evaluator must 
know the criteria on which decisions are based, and must 
determine the critical criteria necessary for the information 
to be provided. 

The evaluator and the decision maker must agree on the 
content, scope, and time frame of the information. The 
decision maker must state clearly the population with which 
he is concerned, the degree of confidence he must attach to 
the findings and how generalized the information should be. 
These statements have implications for sampling methods, 
analysis techniques, and resources required. The, decision 
maker must also specify the time scope of the evaluation 
process. 

"Another concern of the formative evaluator is provision 
for the storage and retrieval of data that is beyond the 
specifications of the decision maker in order to be prepared 
for unpredictable information contingencies. This contingency 
information is of great importance xf1;he local ^^^s^^ "]> 
is oriented to the future -aofid: t 
innovation and change • ; Of ; course/ tli^ 
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information would ideally supply the decision maker with 
future alternatives and desirable strate^^ies for their 
attainment. 

Some educators still equate evaluation with research! 
however, evaluating under research conditions means working 
under unrealistic controls, obtaining meaningful results at 
the completion, assiaming or forcing comparisons of subjects, 
and limiting the alternatives to either acceptance or rejec- 
tion of the hypotheses. Fortunately, there are many quasi- 
experimental models that evaluators can use for theYr purposes. 
While experimental in design, they have been adapted to the 
actual situation being analyzed. 

Sources of Information 

According to various evaluation authorities, there are 
three sources of formative evaluation data. (3, p^y^^j 7f 
p. 115) These sources are usualljr referred to as in^^ 
external, and contextual. Internal information is data formu-; 
lated by analyzing the instructional progriam ■being formulated 
in terms of its physical characteristics, content, and format. 
External information is data derived by ^sdyzin^ 
of the educational program or its components upon. the . behaviofs 
of tho3e individuals direct ly connect ed^^^^ 



words, learners , instructors „ parents, .^and otner^^^^^^ ''M^0^^ 
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which the instructional program and its materials are expected 
to function., I^pical examples of contextual sources for an 
instructional program would be learners, psychological and 
sociological characteristics as well as those of the instruc- 
tors, and classroom dynamics and environment. 

The data or information, once gathered and compiled, can 
be sorted into two categories. According to Baker and Alkin 
(1, p. ^08), the first category is information about the 
effects of the educational program. In the formative evalu- 
ation phase these data give a read-out on the effectiveness of 
the program in meeting its goals. The best source data for 
this category are student performance on the criterion set 
of tasks. This would be done usually in light of . comparison 
with student pretest scores. In addition, learner opinions 
concerning program objectives, course content, and instruc- 
tional quality can give important clues as to the program's 
effectiveness. Free^discussion interviews or open ended 
questionnaires can be used for gathering this data. 

The second category of information is that which is 
diagnostic in nature and can be relayed to and used by the 
curriculm developer in altering the program during the forma- 
tive stage. The data should be so used that the probability 
of program success increases with each additional trial, and 
program deficiencies are remedied and decreased as discovered. 

Baker and Alkin (1 , p. 7^09) suggested several 'tlclmique^ .V 

gather ing these data . Forv^xanpie> , expiert ;ji\Jld^ 
-mended in the period when product spec iil^ 
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developed. The utilization of a panel of judges can be prof- 
itable in two areas. First, they can render expert judgment 
through the review process. Materials can be submitted to 
them as they are prepared and their suggestions can be in- 
corporated during the critical formative stages. Second, as 
a panel of judges, they can be convened as a group. In this 
situation, Delphi techniques and scenario writing can be 
utilized to draw upon their pooled resources. Both techniques 
are based conceptually upon the well-known process called 
brainstorming. 

Program response data is also recommended for gathering 
diagnostic information. (1) This involves, for example, a 
study of the pattern of errors made by learners on practice 
exercises. In addition, any records of student behavior that 
give clues from his pre-program experiences should be analyzed 
The students responses to particular instructional phases 
and devices can also be monitored for data. In fact, pro- 
gram-critique data derived from student opinions have proven 
valuable. Three ways are suggested for obtaining this infor- 
mation (1 ) « ' , 

Pirst, learners may be instructed to circle 
difficult words, write comments, or place 
question marks in the materials that they 
are exposed to. If materials do not present 
printed material, then periodic, brief for- 
mative checks might be made at regular in- 
tervals, where the learner is asked to. pro- 
vide answers to questions similar to those 
listed above. Or single learners may be . 
queried as they proceed through the progrtam. 
A third source is :a post-instructional diag-^ 
nostic questiorinaire, where students are" > 
asked to give their analyses and suggestions 
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for revision of the sequence. An interview 
can also provide information that should 
enable the developer to identify particular 
aspects for revision. 

Formative Evaluation Concepts 

The book produced by Stufflebeam and the Phi Delta Kappa 
National Study Committee on Evaluation (6) covers the entire 
evaluation spectrrom. Its basic contents apply to both forma- 
tive and summative evaluation efforts. This final section 
gives an overview of the CIPP (context, input, process, prod- 
uct) model as presented in the book and briefly summarizes 
its major stages as related to formative evaluation. 

Continuous evaluation is the key to the CIPP model. It. 
includes several basic activities! 

• Creating a proper climate for evaluation. 

« Training personnel to become sensitive to 
decision making stimuli. 

• Emphasizing decision making for all pro- 
ject personnel. 

• Collection of data for feedback. 

• Management of operations. 

Directors of evaluation efforts have the task of creating 
a proper. climate for evaluation • Therefore! chief ^^valuators 
are not data collecting technicians but managers "proy 
plans for evaluation activities • Evaluatorslm^^ 
personnel to help and must also train them 
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establish criteria and methodologies for directing the evalu- 
ation effort, delineating the kinds of data needed, the manner 
in which they will be collected, the personnel who will col- 
lect the information, and the proper manner of utilizing the 
data to solve problems and offer alternatives. Formative 
evaluators should continue to be involved with the program 
throughout its progress toward the final product. CIPP refers 
to the various stages of a program's progression to completion 

The context, input, process, and product (CIPP) stages 
of program evaluation are partitioned as follows i 



. Context evaluation is concerned with the 
setting or the environment in which educa- 
tion is taking place. It includes the com- 
munity, children, teachers, buildings, and 
present approaches used for teaching.^ The 
basic question of concept evaluation is — 
What are the needs that should be satisfied? 

Input evaluation translates the decision 
made under context to specific objectives 
and procedures for achieving the general 
objectives implied in the context decisions. 
The basic question of input "evaluation is 
What are the best means to solve the need 
or the problem? 

. Process evaluation is an examination of 
the program as it functions to see if the ^ 
program plan meets expectations or . if modi- 
fications should be made in operations. 
This kind of evaluation not only wants to 
see whether everything works or not, but 
also wants to provide an avenue for makin^g 
modifications wherever the need indicates. 

Product evaluation analysaes the results to 
see if the objectives of the project ^^^^ 
me t . Some evaluators: refer. ,tOiJ;his^^^ . 
assessment as suimnatiye 6^ 



The comprehensiveness of the CIPP Model jmakes^^i^ 
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applicable to a wide range of situations and contingencies • 
It has been utilized in its totality by various municipal 
school systems. Smaller schools, with limited budgets for 
evaluation can profit from the model's adaptability. They 
can utilize any of Its stages for srry given evaluation task 
that might present Itself. The only limitations imposed upon 
an evaluation effort's effectiveness are the common limiters 
of time, money, and expertise. Granted the proper amounts 
of resources, an evaluation effort's success hinges upon the 
proper relationships established among the evaluation, deci- 
sion making, and educational processes. 

An overview of educational evaluation theory in general 
and its formative phase in specific was presented. Educational 
evaluation was defined as was its phases of formative and 
summatlve evaluation and its classifications of external and 
internal evaluation. Each phase and classification were 
discussed largely in the context jof l^^ In 
addition, various key aspects of formative evaluation were 
discussed — key problems, sources of Infomation, activity 
phases, and types of formative evaluation. 
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